/etc/fstab files tend to be an unreadable mess of unaligned fields.

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/mapper/benz-root /               ext3    errors=remount-ro 0       1
/dev/sda1       /boot           ext3    defaults        0       2
/dev/mapper/benz-home /home           ext3    defaults        0       2
/dev/mapper/benz-swap_1 none            swap    sw              0       0
newton:/home        /nfs        nfs defaults,soft,intr,users    0 0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto     0       0

Let's remove some whitespace in the third line:

#<filesystem> <mountpoint>   <type>  <options>       <dump>  <pass>

And then pipe everything from line 3 to the end through column -t:

# /etc/fstab: static file system information.
#
#<filesystem>            <mountpoint>   <type>       <options>                 <dump>  <pass>
proc                     /proc          proc         defaults                  0       0
/dev/mapper/benz-root    /              ext3         errors=remount-ro         0       1
/dev/sda1                /boot          ext3         defaults                  0       2
/dev/mapper/benz-home    /home          ext3         defaults                  0       2
/dev/mapper/benz-swap_1  none           swap         sw                        0       0
newton:/home             /nfs           nfs          defaults,soft,intr,users  0       0
/dev/scd0                /media/cdrom0  udf,iso9660  user,noauto               0       0

Thanks to SP8472 for bringing this to my attention.

Just discovered (thanks to XTaran): df -T -- show file system type.

$ df -Th
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/mapper/benz-root
              ext3    6.6G  4.9G  1.4G  79% /
tmpfs        tmpfs    1.6G     0  1.6G   0% /lib/init/rw
udev         tmpfs    1.6G  228K  1.6G   1% /dev
tmpfs        tmpfs    1.6G     0  1.6G   0% /dev/shm
/dev/sda1     ext3    236M   21M  203M  10% /boot
/dev/mapper/benz-home
              ext3    135G   82G   53G  61% /home
newton:/home   nfs    459G  145G  291G  34% /nfs

paste is one of those tools nobody uses [1]. It puts two file side by side, line by line. One application for this came up today where some tool was called for several files at once and would spit out one line by file, but unfortunately not including the filename.

$ paste <(ls *.rpm) <(ls *.rpm   xargs -r rpm -q --queryformat '% name  \n' -p)

[1] See "J" in The ABCs of Unix

$ for i in  seq 1 40  ; do ./something $i ; done
$ for i in $(seq 1 40) ; do ./something $i ; done

seq is nice for that, but the syntax feels a bit hard to type. If you don't need sh compatibility (read: in bash/zsh), try:

$ for i in  1..40  ; do ./something $i ; done

PS: no spaces allowed PS 2: another useful feature is prefixes/suffixes as in "touch 1..40 .txt"

When there a security update, just logging in to every host and issuing "apt-get install $pkg" doesn't work as the package might not be installed there. The fact that apt-get doesn't understand "apt-get upgrade $pkg" has bugged me for a long time. Recent aptitude versions support that, but that's not part of Lenny. Here's a shell function that does the trick:

upgrade ()  
        if [ "$*" ] ; then
                set -- $(dpkg -l "$@"   grep ^ii   awk '  print $2  ')
                if [ "$*" ] ; then
                        echo "apt-get install $@"
                        sudo apt-get install "$@"
                else
                        echo "Nothing to upgrade"
                fi
        else
                sudo apt-get upgrade
        fi
 

One application is upgrading a lot of hosts when logged in with clusterssh.

I'm a fan of LC_MESSAGES=de_DE and localized web pages -- almost. Unfortunately, the wording of many translations in Debian often drives me nuts. This applies mostly to web pages and package descriptions. Apologies to all hard-working translators -- the percentage of good translations does not outweigh the "wtf" cases for me. Still, I didn't want to switch to English for all the web. Rhonda pointed me to a Firefox ^W Iceweasel plugin: HeaderControl. Now *.debian.org is English for me (reading the constitution in German doesn't solve DAM problems anyway). Yay!

Sometimes remote accounts are only reachable with a series of ssh/su/sudo commands. Using ProxyCommands in .ssh/config works for simple cases, but not with several hops, or if passwords have to be entered. The belier tool takes as input a series of user@hostname strings and will produce an expect script that does the actual login work.

$ cat input
user@host1 root pw1
user2@host2 pw2
$ bel -e input
$ cat host2.sh
#!/usr/bin/expect -f
set timeout 10
spawn ssh -o NoHostAuthenticationForLocalhost=yes -o StrictHostKeyChecking=no  user@host1
expect -re  "(% # \\$) $"
send -- "su - root\r"
expect ":"
send -- "pw1\r"
expect -re  "(% # \\$) $"
send -- "ssh -o NoHostAuthenticationForLocalhost=yes -o StrictHostKeyChecking=no  user2@host2\r"
expect -re  @[^\n]*: 
send -- "pw2\r"
expect -re  "(% # \\$) $"
interact +++ return

The generated host2.sh script uses ugly ssh options, but is easily edited. Now I need the same thing for scp... Update: Carl Chenet, belier's author, kindly pointed me to the documentation which has examples how belier can set up ssh tunnels to copy files.

Debian and Ubuntu have a set of official membership roles that can be granted to regular contributors. Those roles come with rights that enable the contributors to do their work and to participate in the project governance (elections and other official decision-making processes). It s also a way for the distributions to acknowledge the work done: most contributors are proud of the status they reached. The membership structure plays an important role in the development of a distribution: it defines the kind of contributors that are welcome in the project, it sets expectations of the project towards its contributors and defines their rights. In the end, this shapes the project s ability to recruit new contributors to keep the project alive and kicking. This article introduces the existing statuses in Debian and Ubuntu, and defines the sometimes confusing jargon associated with them. The Debian Case Debian only has two types of official members: Debian Developers (DD) and Debian Maintainers (DM). The rights of the developers are codified in the Debian Constitution while those of the maintainers have been defined in a general resolution of 2007. The Debian Maintainer status is still mostly documented in a wiki page. The integration of this new status in Debian s official processes has been slow to come largely because it was introduced at that time without enough negotiation with the involved parties. Nowadays, it is preferred that people get the DM status before applying for DD. DM is a very limited role: maintainers can only upload packages that already have their name on them (either in the Maintainer or Uploaders field) and a specific flag (DM-Upload-Allowed: yes) that only Debian Developers can add. They have no other rights and limited access to Debian s resources. Besides those official roles, there are also maintainers of packages that have no official status within Debian except that they are listed in the Maintainer field of the package. They are doing the maintenance work but all uploads are done by a Debian Developer after verification of the work done (this is called sponsorship and is the only way to start with official packaging work). Once the DD trusts the maintainer, the developer will typically ask the maintainer to apply for DM status in order to be relieved from the sponsorship work. In the end, that makes three different kind of package maintainers and a lot of confusion when you discuss membership issues in particular when the New Maintainer process is the path that you follow to become a Debian Developer. Don t be fooled by the names when reading Debian s documentation! The Ubuntu Case Ubuntu had, from the start, an official Ubuntu Member status that includes all contributors: developers of course, but also documentation writers, artists, translators, etc. This status notably grants the right to vote in elections of the Community Council, the right to participate on Planet Ubuntu, and the @ubuntu.com email alias. For developers, the situation is more complicated: the wiki page lists no less than five different statuses. Initially, developers were split between Ubuntu Core Developers and the MOTU (Masters Of The Universe). The latter were responsible of the universe/multiverse sections of the archive while the former also had upload rights for the main/restricted sections. But, inspired by the Debian Maintainers concept and facing concrete problems in terms of archive management, they changed their infrastructure to offer more fine-grained control on package uploads. Ubuntu can now grant upload permissions on a package-per-package basis, but it can also delegate the right to grant upload permissions with the same granularity. This lead to the new Per-Package Uploader status which is simply an Ubuntu Member with upload rights on a limited set of packages where they have a specific expertise. The more generic Ubuntu Developers status now encompasses members of various development teams that have been delegated the right to manage upload permissions on a (usually large) package set (the current teams are Ubuntu Desktop, Mythubuntu, Kubuntu, and Edubuntu). Those teams can define their own policy to add new members provided they follow the basic rules defined by the Developer Membership Board (see this wiki page). Ubuntu Contributing Developer is an intermediate status for someone who is not yet ready for one of the other developer statuses but who has still shown enough commitment to be an Ubuntu Member. All those statuses can be obtained in a similar way: you prepare a wiki page listing your past contributions, you collect testimonials from existing members that you have worked with, you add yourself in the agenda of the next meeting of the board (or council) that grants the status that you seek, and you attend the meeting. The members of the board will decide whether you are ready for the status (or not) based on what you provided in the wiki, based on your answers during the meeting (and on a mailing-list for developers), and based on what others have to say about you. The most important boards are usually elected by the community while others are commonly appointed by the community council. Those governance bodies include Canonical employees but not as many as one would expect: two out of eight in the Developer Membership Board, two out of eight in the Community Council, but all six members of the Technical Board. The last figure, while not intended, is not surprising given the high expectations set on potential members of the technical board. Mark, as the founder, is the only person to have a permanent seat on both the Community Council and the Technical Board. Comparison of the Statuses Between Debian and Ubuntu The following table summarizes the rights given to each developer role in the two projects (Put the mouse over the abbreviations to know what they are referring to).

Rights	Debian		Ubuntu
	DM	DD	UM	PPU/UD	MOTU	UCD
Package maintenance via sponsorship	Y	N/A	Y	Y	Y	N/A
Official email alias	-	Y	Y	Y	Y	Y
Participate in votes for members	-	Y	Y	Y	Y	Y
Participate in votes for developers	-	Y	-	Y	Y	Y
Upload rights restricted to pre-approved packages	Y	-	-	Y	-	-
Upload rights restricted to a section of the archive	-	-	-	-	Y	-
Unlimited upload rights	-	Y	-	-	-	Y
Number of contributors (as of 2010-07-27)	117	904	462	27	85	63

Please note that the number of contributors are not 100% accurate for Ubuntu. A contributor can have multiple statuses (direct membership to a launchpad group) granted over time (while gaining experience). The problem has been mostly avoided by calculating differences between number of members of the various groups but it s not perfect and it can t be: some MOTU are also PPU for packages in main and it s legitimate (but I only counted them as MOTU and not as PPU). Another limitation is that members of some administrative teams are included indirectly in many teams and thus appear in the count while they should not. Anyway, this simple table makes it obvious that Ubuntu s structure offers a broader choice of statuses. They acknowledge the work of all contributors from the start while still giving the most critical rights only to those who have proven that they deserve them. Despite this difference, Debian still has a significant advantage in terms of number of developers. That number does not tell the whole story though: the Ubuntu contributors include many Canonical employees (e.g. 36 out of 63 core developers have a @canonical.com email registered on their launchpad account) that are likely to spend more time working on the distribution than the average Debian member. But even if comparing person-hours would be a challenging thought experiment, in practice it s of not much interest if both projects continue to cooperate and if more and more of the contributions flow in both directions. Debian is aware of the shortcomings of its structure. Changes to better accommodate non-packagers have been discussed several times already. The last efforts in that direction were unfortunately perceived as a solution ready to go rather than a proposal to be discussed, and the project got quickly buried by a general resolution (GR). Even if that resolution invited for further discussion and a new proposal, the truth is that when someone s initiative is corrected by way of GR, it usually kills any motivation to go forward. Possible Evolution? On the Ubuntu side, the infrastructural changes were completed recently and they don t expect any further change in the near future. They do plan, however, to expand usage of those new features so that more teams benefit from the possibility to control upload rights on packages that are relevant to them, and so that more individuals developers apply to become Per-Package Uploaders on packages that they know very well. On the Debian side, a recent discussion on the debian-project list brought back the topic of the bad terminology and it was agreed that the New Maintainer process should be renamed into something else ( New Developer process has been suggested). But Christoph Berg Debian Account Manager and hence heavily involved in the New Maintainer Team suggested that Debian would be better off implementing the long-awaited membership changes before trying to update all the documentation. It would certainly imply some more vocabulary updates. Later in the discussion, he confirmed that membership reform is on the top of the TODO list of the new maintainer team (just after the rewrite of the nm.debian.org website). What can be expected from this reform? The following answers are my own guesses based on my experience of Debian, but the project hasn t decided anything yet.

• First of all: a new status for contributors that are not packagers. The tricky part will be defining the process to follow and the rights granted.
• Changes to the technical implementation of the DM status. The current implementation does not allow to give upload rights to a single DM if two are listed in the Uploaders field of a package (and both might not have the same experience for that package). Furthermore, it suffers from annoying restrictions like the inability to upload new binary packages.
• A change of the Debian constitution to integrate those new statuses is almost unavoidable.
• Other more invasive changes have been proposed like replacing the NM process by a simple designation by other DD, but it s unlikely to happen. The NM process can already be greatly simplified by the application manager if the applicant can show good testimonials from other developers and if he has a track record of real contributions (e.g. as witnessed by changelog entries in Debian packages).

Almost two years have elapsed since the previous efforts in that direction, the new maintainer team has recruited new members and is in a general better shape. Hopefully, the next episode of this saga will have a better outcome.

This article was first published in Linux Weekly News. In a comment, Mark Shuttleworth tried to explain how the Ubuntu community is being setup.

10 comments Support my work

Yesterday (Monday), Laura and I got married.

I've always been annoyed about how hard it is to convert seconds-since-epoch to strings. I've always been using "date -d '1970-01-01 + 1234 sec'", but as it turned out, that's wrong because it uses the wrong timezone. Luckily, there's a slick replacement:

$ date -d '@1234'
Do 1. Jan 01:20:34 CET 1970

The right version of the "long" version is:

$ date -d '1970-01-01 UTC + 1234 sec'
Do 1. Jan 01:20:34 CET 1970

Since coreutils 8.1 (in Squeeze, not Lenny), there is a command that simply prints out how many processors (cores, processing units) are available:

$ nproc
2

The use case is obvious:

$ make -j $(nproc)

On a side note, this is gnulib's nproc module wrapped into a C program. If you didn't know gnulib before (it had slipped my attention until recently), it is a library of portability functions and other useful things. Adding it to a project is simply done by calling gnulib-tool and tweaking a few lines in the automake/whatever build scripts. PS: do not use nproc unconditionally in debian/rules. Parse DEB_BUILD_OPTIONS for "parallel" instead.

Lockfiles are usually hard to get right, especially in sh scripts. The best bet so far was "lockfile" included with procmail, but its semantics are pretty weird. (Try to understand what exit code you want, with or without "-!".) Not to mention that failing to clean up the lockfile will stop the next cron run, etc. Since Lenny, util-linux ships "flock". Now you simply say

$ flock /path/to/lockfile /path/to/command

and are done. If you want it non-blocking, add "-n":

$ flock -n /path/to/lockfile /path/to/command

I should probably migrate all my cronjobs to use this.

"stat" is "date +format" for files:

$ stat -c %s ~/me.jpg  # size
520073
$ stat -c %U ~/me.jpg  # owner
cbe

No more parsing of "ls" output or similar hacks. It also shows detailed information about files.

$ stat ~/me.jpg
  File:  /home/cbe/me.jpg 
  Size: 520073          Blocks: 1024       IO Block: 4096   regul re Datei
Device: fe03h/65027d    Inode: 12427268    Links: 1
Access: (0600/-rw-------)  Uid: ( 2062/     cbe)   Gid: ( 2062/     cbe)
Access: 2010-06-06 12:58:07.000000000 +0200
Modify: 2010-04-09 22:38:46.000000000 +0200
Change: 2010-04-26 14:18:00.000000000 +0200

It supports similar features for stat'ing filesystems.

I've always thought about collecting random bits of useful/interesting/cool Unix features and the like. Before I let that rot indefinitely in a text file in my $HOME, I'll post it in a series of blog posts. So here's bit #1: Everyone knows /dev/null, and most will know /dev/zero. But /dev/full was unknown to me until some time ago. This device will respond to any write request with ENOSPC, No space left on device. Handy if you want to test if your program catches "disk full" - just let it write there:

$ echo foo > /dev/full
bash: echo: write error: No space left on device

It has been a while since the last update for tenace, my bridge hand viewer. The highlight in version 0.10 is version 2.0 of the double dummy engine dds which has been updated to support parallel computation in multiple threads. The parscore computation in tenace now uses all available CPU cores. Even my notebook has two CPUs :). More on the technical side, the GUI has been switched to use GtkBuilder which comes with Gtk so there is no external library needed anymore (previously libglade). The looks are pretty much the same as before, though. The previous version 0.9 had added windows support via mingw. I would still appreciate if people could test it and tell me which bits I need to improve.

If you are still using clamav on etch, you might want to upgrade now:

# /etc/init.d/clamav-daemon start  
Starting ClamAV daemon: clamd LibClamAV Warning: ***********************************************************
LibClamAV Warning: ***  This version of the ClamAV engine is outdated.     ***
LibClamAV Warning: *** DON'T PANIC! Read http://www.clamav.net/support/faq ***
LibClamAV Warning: ***********************************************************
/etc/init.d/clamav-daemon: line 240:  5221 Segmentation fault      start-stop-daemon --start -o -c $User --exec $DAEMON
 failed!

Rolling back to yesterday's daily.cld fixes the issue, at least for the segfault.

On Wednesday I got DAM approval and since Saturday late evening I m officially a Debian Developer. Yay! :-) My thanks go to

• Christoph Berg (Myon) whom I know for more than a decade since we studied together, and who s career in Debian was way faster than mine, but who on the other hand probably knows me better than nobody else in Debian which made him the perfect advocate;
• Bernd Zeimetz (bzed) whom I know from my times at DaLUG and who was the friendliest Application Manager I could imagine he s probably also one of the fastest (8 days from application to AM report :-);
• Luca Capello (gismo), who was the most demanding but also most inspiring sponsor I ever had and who became a very good friend after we found each other over my package conkeror.
• Arne Wichmann (Y_Plentyn) for being my first drop-in center for Debian questions (like can I directly dist-upgrade from 2.0 to 3.0? :-);
• Martin Zobel-Helas (zobel) who was always encouraging me to continue exploring new sides of Debian;
• Gerfried Fuchs (Rhonda) just for being there (and for being a package maintainer with good relations to upstream ;-);
• my coworkers at the IT Services Group of the Department of Physics at ETH Zurich, who always found new challenges in Debian for me to solve;
• and all those others who offered to also advocate me (e.g. Otavio Salvador) or sponsored my packages so far (or at least offered to do so), e.g. Alexander Wirt (formorer), Martin F. Krafft (madduck), Robert J rdens (jordens),

As Bernd cited in his AM report, my earliest activity within the Debian community I can remember was organising the Debian booth at LinuxDay.lu 2003, where I installed Debian 3.0 Woody on my Hamilton Hamstation hy (a Sun SparcStation 4 clone). I wrote my first bugreport in November 2004 (#283365), probably during the Sarge BSP in Frankfurt. And my first Debian package was wikipedia2text, starting to package it August 2005 (ITP #325417). My only earlier documented interest in the Debian community is subscribing to the lists debian-apache@l.d.o and debian-emacsen@l.d.o in June 2002. I though remember that I started playing around with Debian 2.0 Hamm, skipping 2.1 (for whatever reasons, I can t remember), using 2.2 quite regularily and started to dive into with Woody which also ran on my first ThinkPad bijou . I installed it over WLAN with just a boot floppy at the Chemnitzer Linux-Tage. :-) Anyway, this has led to what it had to lead to a new Debian Developer. :-) The first package I uploaded with my newly granted rights was a new conkeror snapshot. This version should work out of the box on Ubuntu again, so that conkeror in Ubuntu should not lag that much behind Debian Sid anymore. In other News Since Wednesday I own a Nokia N900 and use it as my primary mobile phone now. Although it s not as free as the OpenMoko (see two other recent posts by Lucas Nussbaum and by Tollef Fog Heen on Planet Debian) it s definitely what I hoped the OpenMoko will once become. And even if I can t run Debian natively on the N900 (yet), it at least has a Debian chroot on it. :-) A few weeks ago, I took over the organisation of this year s Debian booth at FOSDEM from Wouter Verhelst who s busy enough with FOSDEM organisation itself. Last Monday the organiser of the BSD DevRoom at FOSDEM asked on #mirbsd for talk suggestions and they somehow talked me into giving a talk about Debian GNU/kFreeBSD. The slides should show up during the next days on my Debian GNU/kFreeBSD talks page. I hope, I ll survive that talk despite giving more or less a talk saying Jehova! . ;-) What a week.

I'm now a Debian Developer. :) My thanks go to Ben Hutchings, gregor herrmann, Chris Lamb, Christoph Berg, Steve McIntyre, Brad Smith, Jonny Lamb, Chris Boyle, everyone at credativ, and everyone else who helped me with Debian over the last... almost six years? Actually, I've been a DD since the weekend, but have been too absorbed to write about it. In other news, I'm moving house today... I'm more or less packed now. My internet access may be intermittent at home for the next few weeks. (Home is now Southampton, UK.)

Finally, it happened! I became a Debian Developer! I wish to thank everybody involved in this process: from those who helped me in my initial packaging efforts, to who actually created the account. Many thanks to the pkg-perl team for accepting newbies and helping them: props to Damyan Ivanov, Gregor Herrmann, Gunnar Wolf! Thanks to Debian-Med and Debian CLI: they always believed in me, and I had a great time in these teams. A special thanks goes to Andreas Tille, who advocated my NM candidature. Thanks to my two AMs, Bart Martens and Bernd Zeimetz. Even if my NM was long, difficult and not so flawless, it all went good, at the end . A special thanks to Enrico Zini: he was the first DD signing my GPG key, thus allowing me to have more "rights" (being a DM), and experience some more responsibility than I previously had. Thanks to his "Become a DD. NOW!" right after checking my ID in Palermo -- Enrico, it finally happened! Thanks to Christoph Berg, Ron Lee, and Stephen Gran, which handled the final steps of the overall process. And now, dapal is here to break the world. Be warned!

That are only 3 simple words, but their value is hard to underrate. It's very important milestone in my Debian work.

The achievement might not be possible, if there wasn't the help of:
- Vincent Bernat, who thoroughly corrected my first packaging and sponsored my first (and not only) package;
- Dmitry E. Oboukhov, who uploaded many packages for me and also gave some useful advices;
- Roger Leigh, who spent not one day to improve my packaging skills, and also advocated my NM application;
- Enrico Zini, who was my NM Application Manager.
Thank you!

Thanks to people who built my NM procession chain up to creating the account: Bernd Zeimetz (FD), Christoph Berg (DAM), Peter Palfrader (account creation).